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TITLE OF THE INVENTION 

Image Retrieving Apparatus Performing Retrieval Based on Coding 
Information Utilized for Featured Frame Extraction or Feature Values of 
Frames 

5 BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention relates to an image retrieval information 
storing apparatus and an image retrieving apparatus, retrieving an image 
based on the image retrieval information. More specifically, the present 

10 invention relates to an image retrieval information storing apparatus and 
an image retrieving apparatus, using coding information utilized for 
featured frame extraction or feature values of respective frames, as image 
retrieval information. 
Description of the Background Art 

15 When a user retrieves a motion picture, it is important that the user 

recognizes the contents of the motion picture, that is, what is the object 
picked up in the motion picture. Except when the motion picture has a 
text title or description, in most cases it is necessary for the user to confirm 
the contents by playing the whole motion picture. As the number of 

20 motion pictures among which a desired one is to be retrieved increases, it 
becomes harder for the user to play the motion pictures one by one and to 
confirm the contents thereof, and the efficiency of retrieval is quite poor. 
The efficiency of retrieval further lowers when each motion picture is in the 
form of coded compressed data requiring decoding and reproduction before 

25 retrieval. In order to solve this problem, a method of retrieval may be 

effective in which one or a plurality of frames characteristic of the motion 
picture, typically representing the contents of the motion picture are 
extracted and displayed, so that the contents of the motion picture can be 
displayed in schematic manner. 

30 The characteristic frame (hereinafter referred to as featured frame) 

used for motion picture retrieval includes, for example, a scene change 
frame. Japanese Patent Laying-Open No, 9-261648 discloses a method in 
which a scene change frame is extracted, based on prediction mode 



information included in image data, from image data coded in accordance 
with interframe prediction coding. Over one frame, the number of I blocks 
which are coded in accordance with intraframe prediction, P blocks coded in 
accordance with forward interframe prediction, B blocks coded in 
5 accordance with backward interframe prediction and Bi blocks coded in 
accordance with bidirectional interframe prediction are counted. 
Frequency information of respective blocks reflects correlation among 
frames in the motion picture. Thus, based on the frequency information of 
each block, a frame immediately following a scene change (= scene change 

10 frame) is detected. Further, Japanese Patent Laying-Open No. 10-23421 
discloses a method of extracting a scene change frame, based on the change 
in amount of each piece of coding information contained in the coded image 
data. A conventional image retrieval information storing apparatus in 
accordance with the prior art is as shown in Fig. 14. 

15 Referring to Fig. 14, the conventional image retrieval information 

storing apparatus includes a coding information reading unit 1401, a frame 
feature value generating unit 1402 connected to coding information reading 
unit 1401, an index information generating unit 1403 connected to frame 
feature value generating unit 1402, and an index information storing unit 

20 1404 connected to index information generating unit 1403. 

Coding information reading unit 1401 reads, according to Japanese 
Patent Laying-Open No. 9-261648, block by block prediction mode 
information from coded image data, and reads, according to Japanese 
Patent Laying-Open No. 10-23421, respective pieces of coding information 

25 of different types. Frame feature value generating unit 1402 calculates, 
from the various pieces of coding information read by coding information 
reading unit 1401, frequency of respective prediction mode blocks according 
to Japanese Patent Laying-Open No. 9-261648, and calculates accumulated 
value of amount of respective pieces of coding information according to 

30 Japanese Patent Laying-Open No. 10-23421. Index information 

generating unit 1403 determines the scene change frame which is the 
featured frame, based on the frame by frame feature value obtained in 
frame feature value generating unit 1402, that is, frequency information of 
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respective prediction mode blocks according to Japanese Patent Laying- 
Open No. 9-261648> and accumulated amount of respective pieces of coding 
information according to Japanese Patent Laying-Open No. 10-2342 L 
Index information generating unit 1403 outputs positional information of 
5 the scene change frame as index information. Index information storing 
unit 1404 stores the index information output from the index information 
generating unit 1403 as image retrieval information. 

Fig. 15 represents a conventional image retrieving apparatus using 
the image retrieval information stored in this manner. Referring to Fig. 
10 15, the conventional image retrieving apparatus includes an image 

retrieval executing unit 1501 executing retrieval using the image retrieval 
information and the coded image data as inputs and outputting the results 
of retrieval. 

The featured frame includes, in addition to the scene change frame, a 

15 scene change frame based on rapid/slow movement, disclosed in Japanese 
Patent Laying-Open No. 9-284702. According to the invention disclosed in 
Japanese Patent Laying-Open No. 9-284702, a motion vector for detecting a 
scene change frame is obtained from the original image, not from the coded 
information, and the scene change frame is detected based on the total sum 

20 of motion vectors in a frame. 

In the above described conventional image retrieval information 
storing apparatus, positional information of the extracted featured frame is 
obtained, and the positional information only is stored as the index 
information. Therefore, even when a plurality of images which are the 

25 objects of retrieval have mutually different features, only a featured frame 
representing one same feature only can be obtained. If feature frames 
reflecting features of respective images are to be obtained, it is necessary 
for the user to again generate the image retrieval information from the 
coded image data. In the process of retrieval, when featured frames 

30 corresponding to various features of images which are the object of retrieval 
are to be taken out in an interactive manner, re-generation of the image 
retrieval information from the coded image data significantly lowers 
retrieval efficiency. Further, preparation of featured frame indexes 
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corresponding to every possible features for all the image data is not 
practical. Further, it is not possible by the prior art to meet with high 
efficiency a request for extracting a featured frame which has not been 
expected by the provider providing the image retrieval information. 
5 SUMMARY OF THE INVENTION 

The present invention was made to solve the above described 
problems, and its object is to provide an image retrieval information storing 
apparatus capable of immediately meeting a request for switching featured 
frames in the process of retrieval and capable of extracting a featured 
10 frame adaptively. 

Another object of the present invention is to provide an image 
retrieving apparatus capable of high speed image retrieval processing. 

A still further object of the present invention is to provide an image 
retrieving apparatus capable of high speed image retrieval processing and 
15 allowing reduction in capacity used for storing data necessary for retrieval. 

A still further object of the present invention is to provide an image 
retrieving apparatus which can flexibly meet a user's request for extracting 
a featured frame. 

A still further object of the present invention is to provide an image 
20 retrieving apparatus which can flexibly meet a users request for extracting 
a featured frame and which is capable of high speed retrieval processing. 

According to an aspect of the present invention, the image retrieval 
information storing apparatus includes: a coding information reading unit 
reading prescribed coding information which will be information 
25 representing frame features among coded image data; a frame feature 
value generating unit connected to the coding information reading unit, 
generating a frame feature value which is a numerical representation of 
frame features, based on the coding information; and a frame feature value 
storing unit connected to the frame feature value generating unit, and 
30 storing the frame feature value in correspondence with each frame of the 
image data. 

The frame feature value of each frame necessary for extracting a 
featured frame is stored. Therefore, in the process of retrieval, it is 
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possible to meet a request for switching a featured frame immediately, 

whereby adaptive extraction of a featured frame is possible. 

According to another aspect, the image retrieval information storing 

apparatus includes: a coding information reading unit reading prescribed 
5 coding information which will be information representing frame features 

among coded image data; and a coded information storing unit connected to 

the coding information reading unit, and storing coding information in 

correspondence with each frame of the image data. 

The coding information necessary for extracting a featured frame is 
10 stored. Therefore, in the process of retrieval, it is possible to meet a 

request for switching a featured frame immediately, whereby adaptive 

extraction of a featured frame is possible. 

According to a still further aspect, an image retrieval information 

storing apparatus includes: a coding information reading unit reading 
15 prescribed coding information which will be information representing 

frame features among coded image data; a frame feature value generating 

unit connected to the coding information reading unit, and generating a 

frame feature value which is a numerical representation of frame features 

based on the coding information; and a storing unit connected to the coding 
20 information reading unit and the frame feature value generating unit, and 

storing the coding information and the frame feature value in 

correspondence with each frame of the image data. 

The frame feature value frame by frame and the coding information 

necessary for extracting a featured frame are stored. Therefore, in the 
25 process of retrieval, it is possible to meet a request for switching a featured 

frame immediately, whereby adaptive extraction of a featured frame is 

possible. 

In accordance with a further aspect of the present invention, the 
image retrieving apparatus includes: an index information generating unit 
30 receiving a frame feature value which is a numerical representation of 
frame features corresponding to each frame of the coded image data, 
determining a featured frame among the image data based on the frame 
feature value in accordance with a request for extracting a featured frame, 
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and generating index information which is positional information of the 
featured frame; and a image retrieval executing unit connected to the index 
information generating unit, transmitting a request for extracting a 
featured frame to the index information generating unit, receiving the 
5 index information from the index information generating unit, receiving 
image date from outside and outputting a frame specified based on the 
index information. 

The featured frame is extracted based on the frame feature value. 
Therefore, high speed retrieval processing is possible, and capacity used for 

10 storing data necessary for retrieval can be reduced. 

In accordance with a still further aspect, the image retrieving 
apparatus includes: a frame feature value generating unit receiving 
prescribed coding information included in coded image data, which will be 
information representing frame features, and generating a frame feature 

15 value which is a numerical representation of the frame features, based on 
coding information in accordance with a request for extracting a frame 
feature value; an index information generating unit connected to the frame 
feature value generating unit, receiving the frame feature value from the 
frame feature value generating unit, determining a featured frame among 

20 the image data based on the frame feature value in response to a request 
for extracting the featured frame, and generating index information which 
is positional information of the featured frame; and a image retrieval 
executing unit connected to the frame feature value generating unit and 
the index information generating unit, transmitting a request for 

25 extracting frame feature value to the frame feature value generating unit, 
transmitting a request for extracting a featured frame to the index 
information generating unit, receiving index information from the index 
information generating unit, receiving image data from outside, and 
outputting a frame specified based on the index information. 

30 The frame feature value is generated based on the coding 

information. Therefore, when there is a new request for extracting a 
frame feature value, the method of generating the frame feature value for 
extracting the feature frame is changed in the process of retrieval. 



Therefore, it is possible to flexibly meet the user's request for extracting the 
featured frame. 

In accordance with a still further aspect of the present invention, the 
image retrieving apparatus includes: a frame feature value generating unit 
5 receiving prescribed coding information included in coded image data, 

which will be information representing frame features, and generating a 
frame feature value which is a numerical representation of frame features 
based on the coding information in response to a request for extracting the 
frame feature value; an index information generating unit connected to the 

10 frame feature value generating unit, receiving frame feature values from 
the frame feature value generating unit and from the outside, determining 
a featured frame among image data based on the frame feature values in 
response to a request for extracting a featured frame, and generating index 
information which is positional information of the featured frame; and a 

15 image retrieval executing unit connected to the frame feature value 

generating unit and the index information generating unit, transmitting a 
request for extracting frame feature value to the frame feature value 
generating unit, transmitting a request for extracting a featured frame to 
index information generating unit, receiving index information from the 

20 index information generating unit, receiving image data from the outside, 
and outputting a frame specified based on the index information. 

A new featured frame is extracted in accordance with the request for 
extracting the featured frame and the request for extracting frame feature 
value generated in the process of executing image retrieval, and with the 

25 coding information and the frame feature value. Accordingly, flexible 

image retrieval is possible, and high speed retrieving process based on the 
frame feature value is possible. 

The foregoing and other objects, features, aspects and advantages of 
the present invention will become more apparent from the following 

30 detailed description of the present invention when taken in conjunction 
with the accompanying drawings. 
BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 represents a configuration of an image retrieval information 
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storing apparatus in accordance with a first embodiment of the present 
invention. 

Fig. 2 represents internal configuration of a frame feature value 
generating unit. 

5 Fig. 3 represents reference relation of frames in prediction coding 

method. 

Fig. 4 represents a configuration of an image retrieving apparatus in 
accordance with the first embodiment of the present invention. 

Fig. 5 represents an internal configuration of an index information 
1 0 generating unit. 

Fig. 6 represents a configuration of the image retrieval information 
storing apparatus in accordance with the first embodiment of the present 
invention. 

Fig. 7 represents a configuration of the image retrieving apparatus 
15 in accordance with the first embodiment of the present invention. 

Fig. 8 represents a configuration of the image retrieval information 
storing apparatus in accordance with a second embodiment of the present 
invention. 

Fig. 9 represents a configuration of the image retrieving apparatus 
20 in accordance with the second embodiment of the present invention. 

Fig. 10 represents a configuration of the image retrieval information 
storing apparatus in accordance with the second embodiment of the present 
invention. 

Fig. 11 represents a configuration of the image retrieving apparatus 
25 in accordance with the second embodiment of the present invention. 

Fig. 12 represents a configuration of the image retrieval information 
storing apparatus in accordance with a third embodiment of the present 
invention. 

Fig. 13 represents a configuration of the image retrieving apparatus 
30 in accordance with a third embodiment of the present invention. 

Fig. 14 represents a configuration of a conventional image retrieval 
information storing apparatus. 

Fig. 15 represents a configuration of a conventional image retrieving 
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apparatus. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

In the following description of the embodiments of the present 
invention, a scene change frame and a frame in which rapid movement is 
5 locally observed are extracted as featured frames, by way of example, using 
motion vector information and coding prediction mode information among 
coding information included in the image data. The configuration of the 
present invention is also applicable when other coding information is used 
or when other featured frames are to be extracted. 

10 First Embodiment 

Referring to Fig, 1, the image retrieval information storing 
apparatus in accordance with the first embodiment includes a coding 
information reading unit 101, a frame feature value generating unit 102 
connected to coding information reading unit 101, and a frame feature 

15 value storing unit 103 connected to frame feature value generating unit 102. 

Coding information reading unit 101 receives coded image data as an 
input, and reads and outputs desired coding information included in the 
image data. Coding information reading unit 101 in the present 
embodiment extracts and outputs prediction mode information and motion 

20 vector information from the coded image data. 

Frame feature value generating unit 102 processes each piece of 
coding information obtained from coding information reading unit 101, and 
generates and outputs a feature frame value for each frame as a reference 
for determination of a featured frame. 

25 Frame feature value storing unit 103 stores, in a prescribed storing 

location, an output from frame feature value generating unit 102. Details 
of the frame feature value storing unit 103 will be described later. 

Referring to Fig. 2, frame feature value generating unit 102 includes 
a prediction mode counting unit 201 connected to coding information 

30 reading unit 101, frequency information converting unit 202 connected to 
prediction mode counting unit 201, a valid motion vector detecting unit 203 
connected to coding information reading unit 101, a motion vector statistic 
calculating unit 204 connected to coding information reading unit 101 and 
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valid motion vector detecting unit 203, and a statistic information 
converting unit 205 connected to prediction mode counting unit 201 and 
motion vector statistic calculating unit 204. 

The prediction mode information read by coding information reading 
5 unit 101 includes prediction coding method for each frame, and prediction 
method for each coding block in the frame. Prediction mode counting unit 
201 counts the number of blocks coded in accordance with respective 
prediction methods, frame by frame. In a forward prediction coding frame 
(P frame), the number of I blocks coded in accordance with intraframe 

10 prediction and the number of P blocks coded in accordance with interframe 
prediction are counted, respectively. In a bidirectional prediction coding 
frame (frame coded by bidirectional prediction) (B frame ), the number of I 
blocks coded in accordance with intraframe prediction, the number of P 
blocks coded in accordance with forward interframe prediction, the number 

15 of B blocks coded in accordance with backward interframe prediction and 
the number of Bi blocks coded in accordance with bidirectional interframe 
prediction are counted, respectively. 

Further, prediction mode counting unit 201 outputs, as block 
prediction mode frequency information for each frame, the total number Pp 

20 of P blocks for the P frame, and the total number Bf ( = P + Bi ) of forward 
interframe prediction blocks (total number of blocks coded in accordance 
with forward interframe prediction) and total number Bb ( = B + Bi ) of 
backward interframe prediction blocks (total number of blocks coded in 
accordance with backward interframe prediction) for the B frame. 

25 Frequency information converting unit 202 converts these pieces of 

frequency information to attribute information for each frame (in the 
following, the attribute information related to each frame will be referred to 
as the "frame feature value"). In the present embodiment, each frame 
feature value is set such that it represents correlation between the frame of 

30 interest and an immediately preceding frame. 

Referring to Fig. 3, an example of the relation of reference among P 
and B frames at the time of interframe prediction coding will be described. 
A starting point of an arrow represents a frame which is an object of 
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coding/decoding, and the end point of the arrow represents a reference 
frame at the time of coding/decoding. When the reference characters Pp, 
Bfn, and Bbn ( n = 1,2,3) allotted to respective arrows represent the 
aforementioned frequency information, the frame feature value for frame 
B2, that is, frequency information related to correlation between frames Bl 
and B2 should be calculated by a prescribed operation from respective block 
frequency information Pp, Bfn, and Bbn. 

In the present embodiment, the value used as the frame feature 
value of frame B2 is the value of frequency information Bf2 when frequency 
information Bf2 exceeds a certain threshold value, the value of frequency 
information Bbl when frequency information Bf2 does not exceed the 
certain threshold value but frequency information Bbl exceeds the 
threshold value, and otherwise the value of frequency information Bf2 or 
Bbl which is larger. The frame feature value of frame B3 is calculated in 
the similar manner. As to frame B 1, the value of frequency information 
Bf 1 is used as the frame feature value. As to frame P4, the value of 
frequency information Bb3 or Pp which is larger is used as the frame 
feature value. The method of calculating attribute information for each 
frame is not limited to this. Further, the frequency information of the 
coded blocks counted by the prediction mode counting unit 201 may be 
directly regarded as frame feature values. 

The valid motion vector detecting unit 203 detects valid motion 
vector only, from the motion vector information read by coding information 
reading unit 10 1. When a camera is moving while being focused on an 
object, for example, the object and the background of the picked up image 
may have motion vectors very much different in magnitude or direction. 
In such a case, valid motion vector detecting unit 203 extracts only that 
motion vector of an area of interest, for example, the area of the object. 

Possible method of separating the object area from the background 
may include a method in which an average of the motion vectors of the 
entire frame is calculated and the motion vectors out of the average are 
successively extracted, and a method in which areas are divided into two by 
clustering. If the contents of the motion picture come to be more 



complicated as the number of the objects increases, for example, a method 
of valid motion vector detection in accordance with the complexity may be 
used. If the image is not of such a nature, the valid motion vector 
detection may be unnecessary. 
5 Motion vector statistic calculating unit 204 calculates an average 

value of the magnitudes of motion vectors corresponding to P block for the 
P frame over the entire frame, based on the prediction mode information 
obtained by coding information reading unit 101. Motion vector statistic 
calculating unit 204 calculates an average value of magnitudes of motion 

10 vectors corresponding to all the blocks (Bf) coded in accordance with the 
forward interframe prediction and all the blocks (Bb) coded in accordance 
with the backward interframe prediction, for the B frame, over the entire 
frame. The statistics calculated may not be limited to the average value of 
vector magnitudes, and different statistics including average vector may be 

1 5 calculated. Further, the motion vector is normalized considering an 
interval between an object frame and a reference frame at the time of 
calculation. 

Statistic information converting unit 205 converts each of the above 
described motion vector statistics obtained by motion vector statistic 

20 calculating unit 204 to a frame feature value provided for each frame. 
Similar to the frame feature value based on the prediction mode 
information, the frame feature value based on the motion vector 
information in the present embodiment is set such that it indicates the 
motion information between a frame of interest and an immediately 

25 preceding frame. At this time, the coding block frequency information 
obtained by prediction mode calculating unit 201 is referred to in 
calculating the motion vector frame feature value. 

Referring to Fig. 3, statistic information converting unit 205 uses as 
the motion vector frame feature value to be allotted to frame B2, an average 

30 of magnitudes of all motion vectors corresponding to frequency information 
Bf2 and frequency information Bbl, when values Bf2 and Bbl of frequency 
information both exceed a certain threshold value for frame B2. Statistic 
information converting unit 205 uses as motion vector frame feature value, 
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an average of magnitudes of motion vectors corresponding to frequency 
information Bf2, when the value of frequency information Bf2 only exceeds 
the threshold value. Statistic information converting unit 205 uses as 
motion vector frame feature value, an average of magnitudes of motion 
5 vectors corresponding to frequency information Bbl, when frequency 

information Bbl only exceeds the threshold value. Otherwise, statistic 
information converting unit 205 uses 0 as the motion vector frame feature 
value. The motion vector frame feature value allotted to frame B3 is 
calculated in the similar manner. 
10 For frame Bl, statistic information converting unit 205 uses as 

motion vector frame feature value, an average of magnitudes of motion 
vector corresponding to frequency information Bfl when the value of 
frequency information Bfl exceeds a certain threshold value, and otherwise 
0. 

15 For frame P4, statistic information converting unit 205 uses as 

motion vector frame feature value to be allotted to frame P4, an average of 
magnitudes of all motion vectors normalized, corresponding to frequency 
information Bb3 and frequency information Pp, when values of frequency 
information Bb3 and Pp both exceed a certain threshold value. When the 

20 value of frequency information Bb3 only exceeds the threshold values, 
statistic information converting unit 205 uses as motion vector frame 
feature value, an average of magnitudes of the motion vectors 
corresponding to frequency information Bb3. Otherwise, statistic 
information converting unit 205 uses 0 as the motion vector frame feature 

25 value. 

The method of calculating the motion vector frame feature value for 
each frame is not limited to the above. The motion vector statistics 
corresponding to respective blocks described above calculated by motion 
vector statistic calculating unit 204 may be regarded as motion vector 
30 frame statistics frame by frame. 

The frame feature value on prediction mode and the frame feature 
value on motion vector obtained in this manner are output from frame 
feature value generating unit 102 and stored in prescribed storing locations 
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by frame feature value storing unit 103. The storing location of the frame 
feature value may be a location corresponding to the coded image data of 
each frame and, for example, it may be stored as a header information of 
the coded image data of each frame. Alternatively, the frame feature 
value may be stored as a separate data for image retrieval, independent 
from the coded image data. 

Referring to Fig. 4, the image retrieving apparatus in accordance 
with the present embodiment includes an index information generating 
unit 401 connected to an image retrieval executing unit 402, which will be 
described later, and the image retrieval executing unit 402 connected to 
index information generating unit 401. 

Index information generating unit 401 receives as input the frame 
feature value stored in the image retrieval information storing apparatus 
described above, extracts a desired featured frame based on the frame 
feature value of each frame, and generates and outputs as the index 
information, the positional information of the featured frame. 

The process executed in the image retrieval executing unit 402 will 
be described later. 

Referring to Fig. 5, index information generating unit 401 includes a 
scene change frame determining unit 501, connected to image retrieval 
executing unit 402, a scene change frame index generating unit 502 
connected to scene change frame determining unit 501, a motion featured 
frame determining unit 503 connected to image retrieval executing unit 402, 
and a motion featured frame index generating unit 504 connected to motion 
featured frame determining unit 503. 

According to the input frame feature value on prediction mode 
related to coding, a scene change frame is determined by scene change 
frame determining unit 501. In the present embodiment, the scene change 
frame is determined by thresholding the frame feature value on prediction 
modes for coding. By changing the setting of the threshold value or by 
adding a process when an adjacent frame is determined to be the scene 
change frame, the scene change frame to be extracted varies. 

In the prior art, the process of extracting a scene change frame has 
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been independent from the image retrieval process, and the method of 
determining and the like have been fixed. In the present embodiment, the 
threshold value for determining the scene change or the process of 
determination is adaptively changed in response to a request for extracting 
5 a new featured frame output from the image retrieval executing unit 402 in 
the process of image retrieval, as will be described later. Accordingly, the 
desired scene change frames are extracted successively. The scene change 
frame index generating unit 502 generates, as index information, positional 
information of the scene change frame determined by the determining unit 
10 501. 

Using the frame feature value on motion vector as an input, a 
featured frame related to magnitude of motion or the like is determined by 
motion featured frame determining unit 503. In the present embodiment, 
peak position determination is made based on the frame feature values on 

15 motion vectors of neighboring plurality of frames, whereby a frame in 

which local intensive movement is observed is determined. In addition, a 
frame having a magnitude larger than a threshold value may be extracted, 
or a frame of which motion is hardly observed and which intensity of 
motion is on a minimum level, may be extracted as the motion featured 

20 frame. 

In the prior art, the process of extracting a motion featured frame 
has been independent from the image retrieval process, and the method of 
determination and the like have been fixed. In the present embodiment, 
the method of determining motion featured frame related to what motion 

25 feature is of interest is adaptively changed in the process of retrieval, in 
response to a request for extracting a new featured frame output from 
image retrieval executing unit 402 in the process of retrieval, which will be 
described later. Therefore, desired motion featured frames are extracted 
successively. The motion featured frame index generating unit 504 

30 generates, as index information, positional information of the motion 
featured frame determined by the determining unit 503. 

The scene change frame index information and motion featured 
frame index information obtained in this manner are output from index 
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information generating unit 401 and input to image retrieval executing 
unit 402. 

Image retrieval executing unit 402 executes image retrieval based on 
the input featured frame index information and the coded image data. 
Image retrieval executing unit 402 obtains positional information of the 
featured frame from the featured frame index information, and decodes the 
corresponding frame, using the coded image data. The user recognizes the 
contents of the motion picture by thumbnail display (catalogue display) or 
respective featured frames displayed successively, and retrieves a desired 
image. At this time, when it is determined that the featured frame 
applied as index information is not suitable for an image retrieval being 
executed or that the information is insufficient, a request for extracting a 
new featured frame is issued from image retrieval executing unit 402 to 
index information generating unit 40 1 . Receiving the request for 
extracting a new featured frame issued from image retrieval executing unit 
402, index information generating unit 401 changes the method of 
determining featured frames in featured frame determining units 501 and 
503, determines a desired featured frame, and outputs the positional 
information of the featured frame as the index information. The output 
index information is input to image retrieval executing unit 402, and image 
retrieval is newly executed. 

As described with respect to the index information generating unit 
401, the featured frame on motion pictures, such as the scene change frame 
and the featured frame on motion, may be extracted differently dependent 
on the manner of determination, and therefore, physical significance of the 
frame may differ. For example, a frame in which intensive movement is 
observed and a frame in which movement is hardly observed are frames of 
completely different features. 

When motion picture retrieval is executed, whether it is the best to 
display a frame in which intensive movement is observed, or a frame with 
less movement, or a frame having different features on motion as the 
featured frame of the motion picture depends on the motion picture which 
is the object of retrieval. Therefore, it is possible that there is a request for 
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switching the featured frame adaptively in the process of retrieval. In 
such a situation, the image retrieval information storing apparatus 
described above can immediately meet the request for switching the 
featured frame in the process of retrieval, as the image retrieval 
5 information is stored in the form of frame feature values. 

Referring to Fig. 6, it is possible to provide an image retrieval 
information storing apparatus which includes, in addition to the 
configuration of the image retrieval information storing apparatus shown 
in Fig. 1, an index information generating unit 401 generating index 

10 information of the featured frames connected to frame future value 

generating unit 102, and an index information storing unit 601 storing 
index information, connected to index information generating unit 401. In 
this manner, index information of predetermined featured frames are 
separately stored together with the frame feature values. As the frame 

15 feature values and featured frame index information are both stored, a 
request for switching a featured frame in the process of retrieval can be 
immediately met by using the frame feature value, and high speed retrieval 
can be executed by using fixed index information when there is no request 
for switching in the process of retrieval. 

20 Referring to Fig. 7, the image retrieval apparatus corresponding to 

the image retrieval information storing apparatus shown in Fig. 6 includes 
an index information generating unit 401 connected to image retrieval 
executing unit 402, which will be described later, and the image retrieval 
executing unit 402 connected to index information generating unit 401. 

25 Index information generating unit 401 has similar structure as index 

information generating unit 401 described with reference to Figs. 4 and 5. 
Therefore, detailed description thereof is not repeated here. 

Image retrieval executing unit 402 executes image retrieval based on 
the coded image data and index information of featured frames, in the 

30 similar manner as image retrieval executing unit 402 described with 

reference to Fig. 4. At this time, the frame index information is applied 
not only from the index information generating unit 401 but also from 
prescribed storing locations referred by index information storing unit 601 
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of Fig. 6. 

In the present embodiment, the image retrieval information storing 
apparatus and the image retrieving apparatus are different apparatuses 
between which index information and the frame feature values are stored 
and passed. Such information generated by the image retrieval 
information storing apparatus may be directly passed to the image 
retrieving apparatus, and retrieval may be executed on-line. 

Second embodiment 

Referring to Fig. 8, the image retrieval information storing 
apparatus in accordance with the second embodiment includes a coding 
information reading unit 101, and a coding information storing unit 801 
connected to coding information reading unit 101, for storing the coding 
information read by coding information reading unit 101 at a prescribed 
storing location. Coding information reading unit 101 is similar to that 
described in the first embodiment. Therefore, detailed description thereof 
is not repeated. 

The image retrieval information storing apparatus in accordance 
with the first embodiment above is to store the frame feature value of each 
frame. In the image retrieval information storing apparatus in accordance 
with the present embodiment, coding information which is the base for 
extracting the featured frame read by coding information reading unit 101 
is stored in the prescribed storing location by the coding information 
storing unit 801. By this configuration, what is necessary is simply to 
read the coding information at the prescribed location, when the featured 
frame is to be extracted in image retrieval. This improves efficiency as 
compared with successive reading of the original coded image data. The 
location for storing coding information may be a position corresponding to 
the coded image data. For example, it may be stored as header 
information of the coded image data. Alternatively, the coding 
information may be stored independent from coded image data, as different 
data for image retrieval. 

Referring to Fig. 9, the image retrieving apparatus in accordance 
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with the second embodiment includes a frame feature value generating unit 
901 connected to image retrieval executing unit 902, which will be 
described later, and index information generating unit 401 connected to 
frame feature value generating unit 901 and image retrieval executing unit 
5 902, and the image retrieval executing unit 902 connected to index 
information generating unit 401. 

Frame feature value generating unit 901 receives coding information 
stored in image retrieval information storing apparatus, and generates and 
outputs the frame feature value in response to a request for extracting a 

10 frame feature value output from image retrieval executing unit 902. 

Similar to image retrieval executing unit 402 of the first embodiment, 
image retrieval executing unit 902 obtains positional information of 
featured frame from index information of the featured frames, and decodes 
the corresponding frame, using coded image data. At this time, image 

15 retrieval executing unit 902 issues a request for extracting a featured frame 
and a request for extracting frame feature value to index information 
generating unit 401 and the frame feature value generating unit 901, 
respectively. Index information generating unit 401 is similar to that 
described in the first embodiment, and therefore detailed description 

20 thereof is not repeated here. 

In the present embodiment, an image retrieval information storing 
apparatus is implemented in a simple configuration in which only the 
coding information necessary for extracting a featured frame is extracted 
and stored from the original coded image data. Thus, image retrieval 

25 information storing apparatus stores the coding information. Therefore, it 
is possible for the user to change the method of determining a featured 
frame based on the frame feature value, and in addition, to change the very 
method of calculating the frame feature value for extracting the featured 
frame in the process of retrieval. Thus, more flexible image retrieval is 

30 possible. 

As in the first embodiment, referring to Fig. 10, it is possible to 
provide an image retrieval information storing apparatus by adding, to the 
image retrieval information storing apparatus shown in Fig. 8, a frame 
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feature value generating unit 102 connected to coding information reading 
unit 101 and generating a frame feature value for each frame, an index 
information generating unit 401 connected to frame feature value 
generating unit 102 and generating index information of featured frame, 
5 and an index information storing unit 601 connected to index information 
generating unit 401 and storing index information. In this manner, index 
information of a predetermined featured frame is stored separately, 
together with coding information necessary for extracting featured frame. 
When coding information and featured frame index information are 
10 both stored, a request for switching featured frame in the process of 

retrieval can be met by obtaining the frame feature value based on the 
stored coding information, and in a retrieval without a request for 
switching, retrieval at high speed utilizing fixed index information can be 
performed. 

15 Referring to Fig. 11, the image retrieving apparatus corresponding to 

the image retrieval information storing apparatus shown in Fig. 10 
includes a frame feature value generating unit 901 connected to image 
retrieval executing unit 902, which will be described later, an index 
information generating unit 401 connected to frame feature value 

20 generating unit 901 and image retrieval executing unit 902, and the image 
retrieval executing unit 902 connected to index information generating unit 
401. 

Frame feature value generating unit 901 and index information 
generating unit 401 are similar to those described with reference to Fig. 9. 

25 Therefore, detailed description thereof is not repeated here. Image 

retrieval executing unit 902 executes image retrieval based on the coding 
image data and index information of the featured frame, as image retrieval 
executing unit 902 described with reference to Fig. 9. At this time, the 
frame index information is applied not only from the index information 

30 generating unit 401 but also from a prescribed storing location referred by 
index information storing unit 601 of Fig. 10. 

In the image retrieval information storing apparatuses of the first 
and second embodiments, the frame feature value for each frame and 



-20- 



coding information necessary for extracting the featured frame are stored. 
These are not incompatible, and therefore the frame feature value and the 
coding information necessary for extracting the featured frame may be 
stored together. Further, the index information, the frame feature value 
5 and the coding information necessary for extracting the featured frame may 
be all stored and utilized at the time of image retrieval. 

Third Embodiment 

Referring to Fig. 12, the image retrieval information storing 

10 apparatus in accordance with the third embodiment includes a coding 

information reading unit 101, a frame feature value generating unit 102 
connected to coding information reading unit 101, an index information 
generating unit 401 connected to frame feature value generating unit 102, 
a storing information selecting unit 1201 connected to coding information 

15 reading unit 101, frame feature value generating unit 102 and index 

information generating unit 401, an index information storing unit 601, a 
frame feature value storing unit 103 and a coding information storing unit 
801, each connected to storing information selecting unit 1201. 

Coding information reading unit 101, frame feature value generating 

20 unit 102 and index information generating unit 401 are similar to those of 
the first embodiment. Therefore, detailed description thereof is not 
repeated here. Storing information selecting unit 1201 selects and outputs 
one, two or all of index information, frame feature value and the coding 
information necessary for extracting the featured frame, dependent on the 

25 situation. Index information storing unit 60 1, frame feature value storing 
unit 103 and coding information storing unit 801 store the index 
information, the frame feature value and coding information selected by 
storing information selecting unit 1201 at prescribed storing locations, 
respectively. 

30 In the first embodiment, it has been described that the featured 

frame for a motion picture such as a scene change frame and a featured 
frame related to motion may be extracted differently dependent on the 
manner of determination, and that physical meaning of the frame may 
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differ. For example, a frame in which rapid movement is observed and a 
frame in which movement is very small are frames having different 
features. When motion picture retrieval is executed, whether it is 
appropriate to display a frame with rapid movement, or a frame with less 
5 movement, or different frame having different feature related to motion as 
the featured frame for the motion picture depends on the motion picture 
which is the object of retrieval. Therefore, there may be a request for 
switching the condition of extracting the featured frame adaptively in the 
process of retrieval. 

10 In such a situation, the request for switching in the process of 

retrieval may be immediately met when the image retrieval information is 
stored in the form of frame feature value or in the form of coding 
information, by the storing information selecting unit 1201. 

When there is some knowledge of the motion picture which is the 

15 object of retrieval in advance and appropriate featured frame has been 

known, it is possible by storing the image retrieval information in the form 
of index information of the featured frame by storing information selecting 
unit 1201, to reduce the amount of information for image retrieval, and to 
execute retrieval at high speed. Further, when the frame feature value or 

20 the coding information, and the index information of the featured frame are 
to be stored together by the storing information selecting unit 1201, it is 
possible to meet the request for switching in the process of retrieval 
immediately, and to execute retrieval at high speed. 

Referring to Fig. 13, the image retrieving apparatus in accordance 

25 with the third embodiment includes a frame feature value generating unit 
901 connected to image retrieval executing unit 902, which will be 
described later, an index information generating unit 401 connected to 
frame feature value generating unit 901 and image retrieval executing unit 
902, and the image retrieval executing unit 902 connected to index 

30 information generating unit 401. 

Frame feature value generating unit 901 receives, as an input, 
coding information stored in image retrieval information storing apparatus. 
Index information generating unit 401 receives, as an input, frame feature 
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value generated by frame feature value generating unit 901 and the frame 
feature value stored in the image retrieval information storing apparatus. 
Image retrieval executing unit 902 receives as inputs the index information 
stored in image retrieval information storing apparatus and the index 
5 information generated by index information generating unit 401. Index 
information generating unit 401 is similar to that described in the first 
embodiment. Therefore, detailed description thereof is not repeated here. 
Frame feature value generating unit 901 and image retrieval executing 
unit 902 are similar to those described in the second embodiment. 

10 Therefore, detailed description thereof is not repeated here. 

When there is index information, it is possible for the image 
retrieving apparatus to obtain the position of the featured frame based on 
the index information, decode the featured frame based on the input coded 
image data, display the same and to execute the image retrieval. When a 

15 new featured frame is required in the process of image retrieval and there 
is the frame feature value, the image retrieving apparatus extracts the new- 
featured frame based on the frame feature value, and generates the index 
information to be used for image retrieval. 

When it is determined that the frame feature value is inappropriate 

20 and there is coding information, the image retrieving apparatus generates 
a new frame feature value based on the coding information, extracts the 
new featured frame from thus generated new frame feature value, and 
generates index information to be used for image retrieval. 

The advantage of storing both the frame feature value and the 

25 coding information is that image retrieval with such a high degree of 
freedom is possible. 

When there is coding information and not the input frame feature 
value, the request for extracting a new featured frame is directly issued to 
the frame feature value generating unit 901. 

30 In this manner, in the image retrieval information storing apparatus 

in accordance with the third embodiment, the information to be stored is 
selected by the storing information selecting unit 1201 dependent on the 
nature of the image which is the object of retrieval. Therefore, appropriate 
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image retrieval information is stored, and adaptive image retrieval can be 
executed by the image retrieving apparatus. Further it is possible to select 
the stored information by storing information selecting unit 1201 
dependent on the rate of execution of retrieval or on the restriction of 
5 storing capacity. 

As to the capacity of image retrieval information, the index 
information, the frame feature value and coding information have 
capacities larger in this order. For example, when the storage capacity of 
the storing location for image retrieval information is small, only the index 

10 information having small capacity is stored. When the storing capacity is 
sufficiently large and the rate of executing retrieval is slow, all the 
information is stored to ensure degree of freedom of retrieval, and index 
information is fully made use of so as not to lower the execution rate. 

Further, it is possible to limit the image retrieval information to be 

15 stored to only two of the index information, frame feature value and the 
coding information necessary for extracting featured frame, and to select 
the stored information by storing information selecting unit 1201. 

Although the present invention has been described and illustrated in 
detail, it is clearly understood that the same is by way of illustration and 

20 example only and is not to be taken by way of limitation, the spirit and 
scope of the present invention being limited only by the terms of the 
appended claims. 
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WHAT IS CLAIMED IS: 



1. An image retrieval information storing apparatus, comprising: 
a coding information reading unit reading prescribed coding 

information which will be information representing frame feature from 
coded image data; 

a frame feature value generating unit connected to said coding 
information reading unit, and generating a frame feature value which is a 
numerical representation of the frame feature, based on said coding 
information; and 

a frame feature value storing unit connected to said frame feature 
value generating unit, and storing said frame feature value in 
correspondence with each frame of said image data. 

2. The image retrieval information storing apparatus according to 
claim 1, further comprising: 

an index information generating unit connected to said frame feature 
value generating unit, determining a featured frame among said image 
data based on said frame feature value, and generating index information 
which is positional information of said featured frame; and 

an index information storing unit connected to said index 
information generating unit and storing said index information. 

3. The image retrieval information storing apparatus according to 
claim 2, wherein 

said coding information includes prediction mode information and 
motion vector information; and 

said frame feature value generating unit includes 

a prediction mode counting unit connected to said coding information 
reading unit, receiving said prediction mode information and counting 
number of blocks coded in accordance with respective prediction methods 
for each frame, 

a frequency information converting unit connected to said prediction 
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mode counting unit and outputting prediction mode frame feature value 
which is a numerical representation of frame feature, based on the number 
of blocks for each prediction method, 

a motion vector statistic calculating unit connected to said coding 
15 information reading unit, and calculating statistics of said motion vector 
based on said prediction mode information, and 

a statistic information converting unit connected to said prediction 
mode counting unit and said motion vector statistic calculating unit, and 
outputting a motion vector frame feature value which is a numerical 
20 representation of frame feature, base on outputs of said prediction mode 
counting unit and said motion vector statistic calculating unit. 

4. The image retrieving information storing apparatus according to 
claim 1, wherein said coding information includes prediction mode 
information and motion vector information; and 

said frame feature value generating unit includes 
5 a prediction mode counting unit connected to said coding information 

reading unit, receiving said prediction mode information and counting 
number of blocks coded in accordance with respective prediction methods 
for each frame, 

a frequency information converting unit connected to said prediction 
10 mode counting unit and outputting a prediction mode frame feature value 
which is a numerical representation of frame feature, based on the number 
of blocks for each prediction method, 

a motion vector statistic calculating unit connected to said coding 
information reading unit, and calculating said motion vector statistics 
15 based on said prediction mode information, and 

a statistic information converting unit connected to said prediction 
mode counting unit and said motion vector statistic calculating unit, and 
outputting a motion vector frame feature value which is a numerical 
representation of frame feature, based on outputs of said prediction mode 
20 counting unit and said motion vector statistic calculating unit. 
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5. An image retrieval information storing apparatus, comprising: 
a coding information reading unit reading prescribed coding 

information which will be information representing frame feature from 
coded image data; and 

a coding information storing unit connected to said coding 
information reading unit and storing said coding information in 
correspondence with each frame of said image data. 

6. The image retrieval information stoiing apparatus according to 
claim 5, further comprising: 

a frame feature value generating unit connected to said coding 
information reading unit, and generating a frame feature value which is a 
numerical representation of frame feature based on said coding 
information; 

an index information generating unit connected to said frame feature 
value generating unit, determining a featured frame among said image 
data based on said frame feature value, and generating index information 
which is positional information of said featured frame; and 

an index information storing unit connected to said index 
information generating unit and storing said index information. 

7. The image retrieval information storing apparatus according to 
claim 6, wherein 

said coding information includes prediction mode information and 
motion vector information; and 

said frame feature value generating unit includes 

a prediction mode counting unit connected to said coding information 
reading unit, receiving said prediction mode information and counting 
number of blocks coded in accordance with respective prediction methods 
for each frame, 

a frequency information converting unit connected to said prediction 
mode counting unit and outputting a prediction mode frame feature value 
which is a numerical representation of frame feature based on the number 
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of blocks for each prediction method, 

a motion vector statistic calculating unit connected to said coding 
information reading unit and calculating statistics of said motion vector 
based on said prediction mode information, and 

a statistic information converting unit connected to said prediction 
mode counting unit and said motion vector statistic calculating unit, and 
outputting a motion vector frame feature value which is a numerical 
representation of frame feature based on outputs of said prediction mode 
counting unit and said motion vector statistic calculating unit. 

8. An image retrieval information storing apparatus, comprising: 
a coding information reading unit reading prescribed coding 

information which will be information representing frame feature from 
coded image data; 

a frame feature value generating unit connected to said coding 
information reading unit and generating a frame feature value which is a 
numerical representation of frame feature based on said coding 
information; 

a first storing unit connected to said coding information reading unit 
and to said frame feature value generating unit, and storing said coding 
information and said frame feature value in correspondence with each 
frame of said image data. 

9. The image retrieval information storing apparatus according to 
claim 8, further comprising: 

an index information generating unit connected to said frame feature 
value generating unit, determining a featured frame among said image 
data based on frame feature value and generating index information which 
is positional information of said featured frame; 

wherein 

said first storing unit includes a second storing unit connected to 
said coding information reading unit, said frame feature value generating 
unit and said index information generating unit, and storing said coding 
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information and said frame feature value in correspondence with each 
frame of said image data and said index information. 

10. The image retrieval information storing apparatus according to 
claim 9 ? wherein 

said second storing unit includes 

a storing information selecting unit connected to said coding 
information reading unit, said frame feature value generating unit and 
said index information generating unit, and selecting and outputting any of 
said coding information, said frame feature value and said index 
information, 

an index information storing unit connected to said storing 
information selecting unit and storing said index information, 

a frame feature value storing unit connected to said storing 
information selecting unit and storing said frame feature value in 
correspondence with each frame of said image data, and 

a coding information storing unit connected to said storing 
information selecting unit and storing said coding information in 
correspondence with each frame of said image data. 

11. The image retrieval information storing apparatus according to 
claim 8, wherein 

said first storing unit includes 

a storing information selecting unit connected to said coding 
information reading unit and said frame feature value generating unit, and 
selecting and outputting any of said coding information and said frame 
feature value, 

a frame feature value storing unit connected said storing information 
selecting unit and storing said frame feature value in correspondence with 
each frame of said image data, and 

a coding information storing unit connected said storing information 
selecting unit and storing said coding information in correspondence with 
each frame of said image data. 



-29 - 



12. The image retrieval information storing apparatus according to 
claim 10, wherein 

said coding information includes prediction mode information and 
motion vector information; and 

said frame feature value generating unit includes 

a prediction mode counting unit connected to said coding information 
reading unit, receiving said prediction mode information and counting 
number of blocks coded in accordance with respective prediction methods 
for each frame, 

a frequency information converting unit connected to said prediction 
mode counting unit and outputting a prediction mode frame feature value 
which is a numerical representation of frame feature based on the number 
of the blocks for each prediction method, 

a motion vector statistic calculating unit connected to said coding 
information reading unit, and calculating statistics of said motion vector 
based on prediction mode information, and 

a statistic information converting unit connected to said prediction 
mode counting unit and said motion vector statistic calculating unit, and 
outputting a motion vector frame feature value which is a numerical 
representation of frame feature, based on outputs of said prediction mode 
counting unit and said motion vector statistic calculating unit. 

13. The image retrieval information storing apparatus according to 
frame 9. wherein 

said coding information includes prediction mode information and 
motion vector information; and 

said frame feature value generating unit includes 

a prediction mode counting unit connected to said coding information 
reading unit, receiving said prediction mode information and counting 
number of blocks coded in accordance with respective prediction methods 
for each frame, 

a frequency information converting unit connected to said prediction 
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mode counting unit and outputting a prediction mode frame feature value 
which is a numerical representation of frame feature based on the number 
of the blocks for each prediction method, 

a motion vector statistic calculating unit connected to said coding 
information reading unit, and calculating statistics of said motion vector 
based on prediction mode information, and 

a statistic information converting unit connected to said prediction 
mode counting unit and said motion vector statistic calculating unit, and 
outputting a motion vector frame feature value which is a numerical 
representation of frame feature, based on outputs of said prediction mode 
counting unit and said motion vector statistic calculating unit. 

14. The image retrieval information storing apparatus according to 
claim 8, wherein 

said coding information includes prediction mode information and 
motion vector information; and 

said frame feature value generating unit includes 

a prediction mode counting unit connected to said coding information 
reading unit, receiving said prediction mode information and counting 
number of blocks coded in accordance with respective prediction methods 
for each frame, 

a frequency information converting unit connected to said prediction 
mode counting unit and outputting a prediction mode frame feature value 
which is a numerical representation of frame feature based on the number 
of the blocks for each prediction method, 

a motion vector statistic calculating unit connected to said coding 
information reading unit, and calculating statistics of said motion vector 
based on prediction mode information, and 

a statistic information converting unit connected to said prediction 
mode counting unit and said motion vector statistic calculating unit, and 
outputting a motion vector frame feature value which is a numerical 
representation of frame feature, based on outputs of said prediction mode 
counting unit and said motion vector statistic calculating unit. 
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15. An image retrieving apparatus, compiising: 

an index information generating unit receiving a frame feature value 
which is a numerical representation of frame feature corresponding to each 
frame of coded image data, determining a featured frame among said image 
data based on said frame feature value in accordance with a request for 
extracting a featured frame, and generating index information which is 
positional information of said featured frame; and 

a first image retrieval executing unit connected to said index 
information generating unit, transmitting said request for extracting a 
featured frame to said index information generating unit, receiving said 
index information from said index information generating unit, receiving 
said image data from outside, and outputting a frame specified based on 
said index information. 

16. The image retrieving apparatus to claim 15, wherein 

said first image retrieval executing unit includes a second image 
retrieval executing unit connected to said index information generating 
unit, transmitting said request for extracting a featured frame to said index 
information generating unit, receiving said index information from said 
index information generating unit, receiving said image data and index 
information from outside, and outputting a frame specified based on said 
index information received from said index information generating unit 
and from outside. 

17. The image retrieving apparatus according to claim 16, wherein 
said index information generating unit includes 

a frame determining unit receiving said frame feature value and said 
request for extracting a featured frame, comparing said frame feature value 
and a threshold value in accordance with said request for extracting a 
featured frame, and determining said featured frame, and 

an index generating unit connected to said frame determining unit 
and generating index information which is positional information of said 
featured frame. 
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18. The image retrieving apparatus according to claim 15, wherein 
said index information generating unit includes 

a frame determining unit receiving said frame feature value and said 
request for extracting a featured frame, comparing said frame feature value 
and a threshold value in accordance with said request for extracting a 
5 featured frame, and determining said featured frame, and 

an index generating unit connected to said frame determining unit 
and generating index information which is positional information of said 
featured frame. 

19. An image retrieving apparatus, comprising: 

a frame feature value generating unit receiving prescribed coding 
information included in coded image data which wiU be information 
representing frame feature, and generating a frame feature value which is 
5 a numerical representation of the frame feature based on said coding 

information, in accordance with a request for extracting the frame feature 
value; 

an index information generating unit connected to said frame feature 
value generating unit, receiving said frame feature value from said frame 

10 feature value generating unit, determining a featured frame among said 

image data based on said frame feature value in accordance with a request 
for extracting a featured frame, and generating index information which is 
positional information of said featured frame, and 

a first image retrieval executing unit connected to said frame feature 

15 value generating unit and said index information generating unit, 

transmitting said request for extracting a frame feature value to said frame 
feature value generating unit, transmitting said request for extracting a 
featured frame to said index information generating unit, receiving said 
index information from said index information generating unit, receiving 

20 said image date from outside, and outputting a frame specified based on 
said index information. 

20. The image retrieving apparatus according to claim 19, wherein 
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said first image retrieval executing unit includes a second image 
retrieval executing unit connected to said frame feature value generating 
unit and said index information generating unit, transmitting said request 
for extracting a frame feature value to said frame feature value generating 
unit, transmitting said request for extracting a featured frame to said index 
information generating unit, receiving said index information from said 
index information generating unit and from outside, receiving said image 
data from outside, and outputting a frame specified based on said index 
information. 

21. The image retrieving apparatus according to claim 20, wherein 
said index information generating unit includes 

a frame determining unit receiving said frame feature value and said 
request for extracting a featured frame, comparing said frame feature value 
and a threshold value in accordance with said request for extracting a 
featured frame, and determining said featured frame, and 

an index generating unit connected to said frame determining unit 
and generating index information which is positional information of said 
featured frame. 

22. The image retrieving apparatus according to claim 19, wherein 
said index information generating unit includes 

a frame determining unit receiving said frame feature value and said 
request for extracting a featured frame, comparing said frame feature value 
and a threshold value in accordance with said request for extracting a 
featured frame, and determining said featured frame, and 

an index generating unit connected to said frame determining unit 
and generating index information which is positional information of said 
featured frame. 

23. An image retrieving apparatus, comprising: 

a frame feature value generating unit receiving prescribed coding 
information included in coded image data, which will be information 
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representing frame feature, and generating a frame feature value which is 
a numerical representation of the frame feature based on said coding 
5 information, in accordance with a request for extracting a frame feature 
value; 

an index information generating unit connected to said frame feature 
value generating unit, receiving said frame feature value from said frame 
feature value generating unit and from outside, determining a featured 
10 frame among said image data based on said frame feature value in 

accordance with a request for extracting a featured frame, and generating 
index information which is positional information of said featured frame; 
and 

a first image retrieval executing unit connected to said frame feature 
15 value generating unit and said index information generating unit, 

transmitting said request for extracting a frame feature value to said frame 
feature value generating unit, transmitting said request for extracting a 
featured frame to said index information generating unit, receiving said 
index information from said index information generating unit, receiving 
20 said image data from outside, and outputting a frame specified by said 
index information. 

24. The image retrieving apparatus according to claim 23, wherein 
said first image retrieval executing unit includes a second image 

retrieval executing unit connected to said frame feature value generating 
unit and said index information generating unit, transmitting said request 
5 for extracting a frame feature value to said frame feature value generating 
unit, transmitting said request for extracting a featured frame to said index 
information generating unit, receiving said index information from said 
index information generating unit and from outside, receiving said image 
data from outside, and outputting a frame specified based on said index 
10 information. 

25. The image retrieving apparatus according to claim 24, wherein 
said index information generating unit includes 
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a frame determining" unit receiving said frame feature value and said 
request for extracting a featured frame, comparing said frame feature value 
and a threshold value in accordance with said request for extracting a 
featured frame, and determining said featured frame, and 
5 an index generating unit connected to said frame determining unit 

and generating index information which is positional information of said 
featured frame. 

26. The image retrieving apparatus according to claim 23, wherein 

said index information generating unit includes 

a frame determining unit receiving said frame feature value and said 
request for extracting a featured frame, comparing said frame feature value 
5 and a threshold value in accordance with said request for extracting a 
featured frame, and determining said featured frame, and 

an index generating unit connected to said frame determining unit 
and generating index information which is positional information of said 
featured frame. 
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ABSTRACT OF THE DISCLOSURE 

An image retrieval information storing apparatus capable of 
extracting a featured frame adaptively in the process of retrieval includes a 
coding information reading unit reading prescribed coding information 
5 which will be information representing frame feature among coded image 
data, a frame feature value generating unit connected to the coding 
information reading unit and generating a frame feature value which is a 
numerical representation of the frame feature based on the coding 
information, and a frame feature value storing unit connected to the frame 
10 feature value generating unit and storing the frame feature value in 
correspondence with each frame of the image data. 
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